BY-COVID - WP5 - Baseline Use Case: COVID-19 vaccine effectiveness assessment

Survival analysis

Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted.


Survival (time-to-event)

The probability of not getting infected by SARS-CoV-2 beyond a certain time after onset of follow-up (survival function, estimated using the Kaplan-Meier estimator) is reported for different periods.

Strata Time Number at risk Cumulative sum of number of events Cumulative sum of number censored Survival Std. error Cumulative hazard Std. error cumulative hazard
Not fully vaccinated 0 38785 0 0 1.0000 0.00000 0.0000 0.00000
Not fully vaccinated 100 13487 269 25135 0.9881 0.00077 0.0119 0.00078
Not fully vaccinated 200 4428 1046 33416 0.8974 0.00331 0.1081 0.00369
Not fully vaccinated 300 3002 1591 34192 0.7618 0.00606 0.2717 0.00794
Not fully vaccinated 400 2377 1690 34770 0.7366 0.00636 0.3054 0.00863
Not fully vaccinated 500 445 1696 36664 0.7331 0.00651 0.3102 0.00887
Not fully vaccinated 600 2 1696 37087 0.7331 0.00651 0.3102 0.00887
Fully vaccinated 0 38785 0 0 1.0000 0.00000 0.0000 0.00000
Fully vaccinated 100 13445 314 25130 0.9849 0.00089 0.0152 0.00090
Fully vaccinated 200 4298 1299 33288 0.8706 0.00367 0.1385 0.00421
Fully vaccinated 300 2992 1790 34004 0.7501 0.00597 0.2872 0.00794
Fully vaccinated 400 2412 1875 34528 0.7287 0.00623 0.3162 0.00854
Fully vaccinated 500 462 1876 36468 0.7283 0.00624 0.3167 0.00856
Fully vaccinated 600 2 1876 36907 0.7283 0.00624 0.3167 0.00856


Median survival time

The median survival time is the time corresponding to a probability of not obtaining a SARS-CoV-2 infection probability of 0.5. (if NA, the probability of not obtaining a SARS-CoV-2 infection did not drop below 50%)

Characteristic Median survival (95% CI)
fully_vaccinated_bl
    FALSE — (—, —)
    TRUE — (—, —)
Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate). The Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate and accounts for clustering within individuals (as one individual can be re-sampled as control).

A hazard ratio (HR) is computed for the covariate ‘fully_vaccinated_bl’. A hazard can be interpreted as the instantaneous rate of SARS-CoV-2 infections in individuals that are at risk for obtaining an infection (Cox proportional hazards regression assumes stable proportional hazards over time). A HR < 1 indicates reduced hazard of SARS-CoV-2 infection when having completed a primary vaccination schedule whereas a HR > 1 indicates an increased hazard of SARS-CoV-2 infection.

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.111 0.034 0.089 0.211 1.117 (0.923, 1.312)

The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 11.00129 1 0.0009104850
Wald test 1.57000 1 0.2107969448
Score (logrank) test 10.99772 1 0.0009122426
Robust score test 1.63271 1 0.2013285629

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Arm Measure Estimate SE CI.lower CI.upper
fully_vaccinated_bl==FALSE RMST 325.159 0.919 323.357 326.961
fully_vaccinated_bl==TRUE RMST 321.545 0.940 319.702 323.389
fully_vaccinated_bl==FALSE RMTL 39.841 0.919 38.039 41.643
fully_vaccinated_bl==TRUE RMTL 43.455 0.940 41.611 45.298
Measure Estimate CI.lower CI.upper p_value
RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.614 -6.191 -1.036 0.006
RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.989 0.981 0.997 0.006
RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.091 1.025 1.160 0.006
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the vaccination schedule.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the vaccination schedule.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the administered vaccination schedule. A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘vaccination_schedule_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.053 0.089 0.183 0.771 1.055 (0.677, 1.433)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-BP 0.056 0.098 0.183 0.760 1.058 (0.678, 1.437)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ 0.078 0.192 0.349 0.824 1.081 (0.341, 1.821)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)MD-MD 0.116 0.124 0.221 0.599 1.123 (0.637, 1.608)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 11.952260 4 0.01770982
Wald test 1.940000 4 0.74735611
Score (logrank) test 11.943612 4 0.01777555
Robust score test 1.996748 4 0.73635710

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Vaccination_schedule Arm Measure Estimate SE CI.lower CI.upper
BP-BP fully_vaccinated_bl==FALSE RMST 325.833 1.109 323.659 328.007
BP-BP fully_vaccinated_bl==TRUE RMST 323.221 1.124 321.018 325.423
BP-BP fully_vaccinated_bl==FALSE RMTL 39.167 1.109 36.993 41.341
BP-BP fully_vaccinated_bl==TRUE RMTL 41.779 1.124 39.577 43.982
MD-MD fully_vaccinated_bl==FALSE RMST 325.768 2.336 321.190 330.346
MD-MD fully_vaccinated_bl==TRUE RMST 319.131 2.451 314.328 323.935
MD-MD fully_vaccinated_bl==FALSE RMTL 39.232 2.336 34.654 43.810
MD-MD fully_vaccinated_bl==TRUE RMTL 45.869 2.451 41.065 50.672
AZ-AZ fully_vaccinated_bl==FALSE RMST 318.984 2.670 313.751 324.216
AZ-AZ fully_vaccinated_bl==TRUE RMST 315.005 2.877 309.366 320.643
AZ-AZ fully_vaccinated_bl==FALSE RMTL 46.016 2.670 40.784 51.249
AZ-AZ fully_vaccinated_bl==TRUE RMTL 49.995 2.877 44.357 55.634
JJ fully_vaccinated_bl==FALSE RMST 331.275 4.497 322.462 340.088
JJ fully_vaccinated_bl==TRUE RMST 319.813 4.593 310.811 328.815
JJ fully_vaccinated_bl==FALSE RMTL 33.725 4.497 24.912 42.538
JJ fully_vaccinated_bl==TRUE RMTL 45.187 4.593 36.185 54.189
Vaccination_schedule Measure Estimate CI.lower CI.upper p_value
BP-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -2.612 -5.708 0.483 0.098
BP-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.992 0.983 1.001 0.098
BP-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.067 0.988 1.152 0.098
MD-MD RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -6.636 -13.272 0.000 0.050
MD-MD RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.980 0.960 1.000 0.050
MD-MD RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.169 0.999 1.368 0.051
AZ-AZ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.979 -11.672 3.713 0.311
AZ-AZ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.988 0.964 1.012 0.311
AZ-AZ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.086 0.926 1.275 0.310
JJ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -11.463 -24.060 1.135 0.075
JJ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.965 0.929 1.004 0.075
JJ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.340 0.965 1.861 0.081
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the NUTS3 residence area.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the NUTS3 residence area.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the area of residence (NUTS3 level). A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘residence_area_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.164 0.092 0.235 0.486 1.178 (0.635, 1.721)
fully_vaccinated_blTRUE:strata(residence_area_cd)732 -0.051 0.125 0.328 0.875 0.95 (0.34, 1.56)
fully_vaccinated_blTRUE:strata(residence_area_cd)733 -0.064 0.100 0.258 0.805 0.938 (0.465, 1.412)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 11.311765 3 0.01015415
Wald test 1.630000 3 0.65205542
Score (logrank) test 11.298380 3 0.01021714
Robust score test 1.705346 3 0.63574596

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Residence_area Arm Measure Estimate SE CI.lower CI.upper
733 fully_vaccinated_bl==FALSE RMST 326.329 1.061 324.249 328.409
733 fully_vaccinated_bl==TRUE RMST 323.213 1.094 321.069 325.357
733 fully_vaccinated_bl==FALSE RMTL 38.671 1.061 36.591 40.751
733 fully_vaccinated_bl==TRUE RMTL 41.787 1.094 39.643 43.931
731 fully_vaccinated_bl==FALSE RMST 324.847 2.596 319.759 329.935
731 fully_vaccinated_bl==TRUE RMST 319.555 2.526 314.605 324.506
731 fully_vaccinated_bl==FALSE RMTL 40.153 2.596 35.065 45.241
731 fully_vaccinated_bl==TRUE RMTL 45.445 2.526 40.494 50.395
732 fully_vaccinated_bl==FALSE RMST 318.840 2.589 313.766 323.914
732 fully_vaccinated_bl==TRUE RMST 314.860 2.655 309.657 320.064
732 fully_vaccinated_bl==FALSE RMTL 46.160 2.589 41.086 51.234
732 fully_vaccinated_bl==TRUE RMTL 50.140 2.655 44.936 55.343
Residence_area Measure Estimate CI.lower CI.upper p_value
733 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.116 -6.103 -0.128 0.041
733 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.990 0.981 1.000 0.041
733 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.081 1.003 1.164 0.041
731 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -5.292 -12.391 1.807 0.144
731 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.984 0.962 1.006 0.144
731 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.132 0.958 1.338 0.146
732 RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.980 -11.247 3.288 0.283
732 RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.988 0.965 1.010 0.283
732 RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.086 0.934 1.263 0.284